Imaging device and methods to derive an image on a solid phase

ABSTRACT

An imaging device ( 46 ) comprises a carrier stage ( 12   a ) for carrying a sample slide ( 14   a ) including a micro-array of cellular binding event samples, a linear light source ( 37   a ) for illuminating the sample slide ( 14   a ), and a motor drive ( 16   a ) for moving the carrier stage ( 12   a ) relative to the sample slide ( 14   a ) such that successive portions of the sample slide ( 14   a ) are illuminated by the light source ( 37   a ). A digital opitical line scan camera system ( 44   a ) is disposed such that, in use, it captures substantially only the successive portions of light rays ( 40   a ) which emerge from the sample slide at an offset angle relative to light rays ( 42   a ) from light source transmitted through and emerging from the sample slide ( 14   a ) to generate a series of linear dark field images arranged to be reconstructed into a composite image of the sample slide or array of samples.

FIELD OF THE INVENTION

The present invention relates broadly to an imaging device and to amethod of deriving an image of samples on a transparent solid phase suchas a sample slide.

BACKGROUND OF THE INVENTION

The analysis of samples such as cells, for example those obtained from apatient, bound to an arrangement of binding partners, such as a proteinmicro-array on a glass slide has been proposed as a diagnostic tool.

Similarly, the analysis of the presence of fluorescent marketsindicative of the presence of particular molecules such as proteins in asample has been proposed as a diagnostic tool.

It is desirable to provide a device for capturing digitised patterns ofsuch samples to facilitate the use and implementation of such adiagnostic tool. It is further desirable to provide a device which canbe made readily available for widespread usage over a distributednetwork of pathology laboratories and research facilities.

In at least preferred embodiments, the present invention seeks toprovide a compact imaging device and a method of deriving an image ofsamples on a sample slide suitable for implementation of such adiagnostic tool.

SUMMARY OF THE INVENTION

According to a first aspect of the invention there is provided animaging device comprising:

-   -   a carrier stage for carrying a sample slide,    -   a light source for illuminating the sample slide, said sample        slide including an array of samples,    -   drive means for moving the carrier stage relative to the sample        slide such that successive portions of the sample slide are        illuminated by the light source;    -   a digital optical camera system disposed such that in use, it        captures substantially only said successive portions of light        rays which emerge from the sample slide at an offset angle        relative to light rays from light source transmitted through and        emerging from the sample slide to generate a series of partial        images arranged to be reconstructed into an image of the sample        slide or array of samples.

Preferably, the light source is a linear light source arranged to emit asubstantially narrow beam, whereby successive portions of the sampleslide that are illuminated are band-like portions, and whereby theseries of partial images are linear images.

Conveniently, the digital optical camera system is disposed such that,in use, it receives substantially only light rays which are diffractedor otherwise deflected at said array of samples on the sample slide.

Typically, the digital optical camera system includes discriminatormeans for preventing light rays which are not diffracted or otherwisedeflected by the sample array from being captured by the camera system.

Advantageously, the discriminator means includes at least one reflectorpositioned to direct diffracted or otherwise deflected light raysemerging from the sample slide at the offset angle towards an imaginglens of the camera system.

The digital optical camera system typically includes a line scan capablecamera capable of sensing a linear image.

The digital optical camera system may be disposed such that, in use,light rays emitted from fluorescent markers on the sample slide arecaptured.

Conveniently, the digital optical camera system is arranged to operatein at least two modes, namely a diffraction or deflection mode, in whichlight rays diffracted or otherwise deflected at the array of samples onthe sample slide are captured by the camera, and a fluorescent mode, inwhich light rays emitted from fluorescent markers on the array ofsamples are captured.

The digital optical camera system may be arranged to operate in thedeflection or diffraction mode when the drive means moves the carrierstage in a first direction and is arranged to operate in the fluorescentmode when the drive means is moves the carrier stage in a seconddirection.

The optical camera system may be arranged to detect light rays in boththe visible and non-visible portions of the spectrum.

Typically the sample comprise an array of cells bound to bindingpartners on the sample slide.

In once form of the invention, the imaging device comprises a samplingcompartment in which, in use, the carrier stage is located, and anelectrical component compartment, wherein the electrical componentscompartment is fluid sealed from the sampling compartment, whereby, inuse, fluid contamination of components inside the electrical componentscompartment from the sampling compartment is inhibited, the carrierstage including a tray element disposed, in use, underneath the sampleslide for collecting fluid spilled from the sample slide.

The imaging device may include an interface unit for interfacing todevices of a group including at least one of an external referencedatabase, an external storage database, an external PC, and an externalprinter.

Advantageously, the partial images and the reconstructed images are darkfield images.

The invention extends to an imaging system including an imaging deviceof the type described above and processor means for processing the imageof the sample slide or array of samples to provide image intensityvalues representative of the array of samples for comparative purposes.

The invention further extends to the processor means for processing theimage of the sample slide or array of samples to provide image intensityvalues representative of the array of samples for comparative purposes.

Advantageously, the processor means is arranged to normalise the imageby using known reference samples on the slide to locate each sample onthe slide and to scale the intensity of each sample.

The processor means is preferably arranged to locate each sample byapplying a reference matrix or grid on the basis of the known referencesample arranged to scale the intensity of the samples within each squarein the grid using the reference samples to establish the range of thescale, and is further arranged to generate a normalised intensity valuesfrom the image.

The invention still further provides a method of deriving an imagerepresentative of samples on a sample slide, the method comprising

-   -   providing a sample slide including an array of samples    -   loading the sample slide onto a carrier stage    -   illuminating at least a portion of the sample slide,    -   moving the carrier stage relative to the sample slide such that        successive portions of the sample slide are illuminated by the        light source; and    -   capturing substantially only successive diffracted or otherwise        deflected portions of light rays which emerge from the sample        slide to generate a series of partial images arranged to be        reconstructed into an image of the array of samples.

Preferably, successive portions of the sample slide that an illuminatedare band-like portions illuminated by ultilising a linear light source,and whereby the series of partial images are captured as linear images.

The method advantageously comprises the step of capturing substantiallyonly light rays diffracted or otherwise deflected by or at samples onthe sample slide.

Preferably, the method further comprises utilising reference samplesdisposed in a manner such that light rays diffracted or otherwisedeflected at the reference samples are captured during the deriving ofthe image, for indicating the biological condition of the sample and/orintensity scaling.

The method may include processing the reconstructed image to arrive at amolecular profile which is comparable with a library of molecularprofile signatures.

The method may further include generating image intensity values foreach sample, generating a contour map of image intensities identifyingimage objects within contour lines, and placing a virtual grid over saidobjects.

The method typically further includes deskewing the image, obtaining anenhanced grid, and calculating X-Y co-ordinates from the enhanced grid.

Preferably, the method includes calculating an averaged correctedintensity for each sample, whereby at least two sets of identicalsamples are provided on the slide, and normalising the intensity dataassociated with each sample on the basis of reference samples andduplicate samples.

The method further extends to an imaging processing method forprocessing an image of the type obtained from the device, as well as toa computer readable medium having stored thereon executable instructionsfor causing a computer to carry out the method,

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention will now be described, byway of example only, with reference to the accompanying drawings.

FIG. 1 is a schematic perspective drawing illustrating an imaging deviceof a first embodiment of the present invention, with parts of thehousing removed and only selected components shown for clarity;

FIG. 2 is a schematic drawing illustrating a different view of theimaging device of FIG. 1, with some of the housing removed and onlyselected components shown for clarity;

FIG. 3 is a schematic drawing illustrating a perspective view of anexample sample slide for use in an imaging device embodying the presentinvention;

FIG. 3A is a top plan view of the sample slide of FIG. 3;

FIG. 4 is a schematic drawing illustrating the optical geometry in theimaging device of FIG. 2;

FIG. 5 is a schematic perspective drawing illustrating a secondpreferred embodiment of an imaging device of the present invention;

FIG. 5A is a top perspective view of a slide tray assembly forming partof the imaging device of FIG. 5;

FIG. 6 is a schematic drawing of a front view of the imaging device ofFIGS. 1 and 5;

FIG. 6A is a functional block diagram of the imaging device of FIG. 5;

FIG. 7 shows an image taken of a sample slide utilising a prototypeimaging device embodying the present invention;

FIG. 8 shows a data array illustrating the information from a sampleslide to be passed to a pattern-matching program, embodying the presentinvention;

FIG. 9 is a schematic drawing illustrating the optical geometry in theimaging device of FIGS. 1 and 3, for a different sample type;

FIG. 10 is a flow chart showing the steps involved in deriving andprocessing an image in an embodiment of an image deriving and processingmethod of the invention;

FIG. 10A shows a step in the imaging processing method of the inventionin which the image is smooth and contoured;

FIG. 10B shows a further processing step in which a virtual grid isarranged to overlay the image; and

FIG. 10C shows a histogram illustrating average intensitiesrepresentative of binding events in the diagnostic marker arrays in theprocessed image of FIG. 10B.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the preferred embodiments described, the present invention providesan imaging device and methods of taking a sequential series of darkfield linear images to construct an offset planar image of a boundcellular array or a sample containing fluorescent markers, suitable foridentifying a molecular profile thereby implementing a diagnostic toolwhich utilises analysis of samples on a sample slide or othertransparent solid phase support media. The offset planar image isdigitally re-assembled to provide a digital array which can be passed onto a pattern matching program or the like for molecular profileidentification.

FIG. 1 shows a schematic diagram of an imaging device 10 embodying thepresent invention. The device 10 comprises a carriage 12 for mounting aslide 14 for analysis of a bound cellular array (102 in FIG. 3) bound onthe slide 14. The carriage 12 comprises two guiding rods 16, 18 ontowhich a slide holder 20 is movably mounted. The holder 20 comprises twobiasing elements in the form of spring members 22, 24 for releasablyreceiving the slide 14.

The imaging device 10 further comprises a magnetic pull stepper drivemechanism of which a pull bar 26 is shown in FIG. 1. As can be moreclearly seen in FIG. 2, the pull bar 26 comprises a magnetic end portion28 for connecting to the holder 20, which is made of a suitable magneticmaterial. The use of a magnetic pull stepper drive mechanism in theexample embodiment has the advantage of providing a readily releasableconnection between the pull bar 26 and the holder 20, to facilitateremoval of the carriage 12 for cleaning or other maintenance purposes ofthe carriage and/or the interior of the imaging device 10.

FIG. 3 shows an isometric drawing of an example of the sample slide 14.The slide 14 comprises a plurality of indents 100 containing localisedbinding events. In particular, each of which typically containsdifferent binding ligands to provide a bound array of binding partners102. The slide 14 is formed from a substantially optically transparentmaterial, such as glass or suitable plastic material such as polystyreneor polycarbonate (Cyrolon TX-V), polyvinyl alcohol, nylon or compositesthereof. Such supporting materials are either untreated or treated withabsorbent or binding enhancing coatings to facilitate the binding ofeach binding partner. In the embodiment, FAST from Schleicher andSchuell BioScience, Inc, of 10 Optical Avenue, Keene N.H. 03431 USAslides were used. These slides are manufactured from high quality glasswith a nitro-cellulose coating. It will be appreciated by a personskilled in the art that there are several chemical and physicalapproaches to secure protein based material on said solid phasematerials. Each slide may contain up to 1,000 or more binding events.

FIG. 3A shows a top plan view of the sample slide 14 for use as adiagnostic tool in diagnosing leukaemia. The array of binding partners102 includes rows of serial calibration dots 104 and 106 for alignmentand intensity correction purposes, and typically covering the fullexpected optical range, from lightest to darkest. Outer peripheral rowsof calibration dots 108 and 110 formed from reference binding partnerssuch as monoclonal antibodies representative of the binding partnersexpected to yield the darkest image. In conjunction, the outerperipheral calibration dots define the spatial boundary of said bindingevents for facilitating image construction. The serial calibration dots104 and 106 may be formed from monoclonal antibodies varying frompredetermined high to low concentrations by progressive dilution. Arraysof a diagnostic markers 112 and 114 are located centrally on the slide,which further includes a sub-array of therapeutic markers 116 and asub-array of diagnostic and QC markers 118. The arrays 112 and 114 aresubstantially identical so they can be used for cross-checking purposes,and the results averaged for a more reliable outcome. Furtherinformation on the slide includes a catalogue number 120, an expiry date122 and a bar code 124 encoding this and other information on the slide.

Returning now to FIG. 1, the pull bar 26 extends through a dividing wall30 within the imaging device 10, which divides the interior of theoptical device 10 into a sampling area 32, and an electrical componentsarea 34. In the example embodiment, the dividing wall 30 is adapted suchthat a fluid seal is created between the sampling area and theelectrical components area, whereby contamination of electricalcomponents inside the electrical components area 34 from the samplingarea 32 is inhibited.

The pull bar 26 extends through the dividing wall 30 via a sealingmember 36, which is adapted to allow movement of the pull bar 26, whilemaintaining a fluid seal between the sampling area 32, and theelectrical components area 34. A drip tray 46 may be provided with thecarriage 12 for collecting fluid that may drip from the slide 14 duringthe analysis.

The imaging device 10 further comprises an LED bracket 37 for emitting asubstantially planar beam of light 38 for taking an image of the boundcellular or protein array 102 (FIGS. 3 and 3A) mounted on the slide 14.The beam 38 is initially reflected by a first mirror (not shown) suchthat it is directed towards the slide from the bottom thereof. Above theslide, a second mirror (not shown) is utilised to direct a portion 40 ofthe initial light beam containing rays which have been diffracted orotherwise deflected at bound binding partners (not shown) towards thedigital camera device. The camera takes successive linear images of theoffset planar light diffracted or otherwise deflected at the slide orsolid phase equivalent, as it moves through the band of light emittedfrom the light source at a speed consistent with image capturecapability of said camera. It will be appreciated by a person skilled inthe art, that through suitable adjustment of the mirror element (notshown) for directing the deflected beam portion the imaging device 10can be adapted in a manner such that substantially only a diffracted orotherwise deflected beam portion 40 is captured in the line scan camera44 for deriving an image of the bound array.

In FIG. 4, a schematic drawing is shown illustrating the opticalgeometry in an example embodiment. The beam 38 emitted from the LEDarray 37 is incident on the slide 14. One portion 40 of the beam 38containing rays deflected at binding partners (not shown) bound on theslide 14 emerges as a deflected portion 40 after the slide 14, at anangle α to an undeflected portion 43. This angle α is typically in therange of 3-5°, and may be empirically determined by adjustment of themirror. Accordingly, through suitable orientation of the mirror 45, itcan be ensured that substantially only the diffracted/deflected oroffset planar portion 40 is directed towards the line scan camera 44,and the undeflected portion is reflected away from the camera, as isshown at 42. It will be appreciated by the person skilled in the artthat at least one other portion (not shown) of the initial beamcontaining light rays deflected or diffracted at binding partners (notshown) bound on the slide 14 is expected to emerge after the slide 14,at an angle (−α) to the undeflected portion 43. The otherdiffracted/deflected or offset portion or portions may collectively,alternatively, or additionally he directed towards the line scan camera44 in different embodiments using a suitable reflecting arrangement orthrough optical variation in the field of view of the camera 44.

It has been recognised by the applicant that the utilisation ofsubstantially only rays that have been diffracted at or otherwisedeflected as a result of binding partners bound on the sample slide 14enables the capturing of a positive image of the bound array. In otherwords, the number of cells or bound partners in individual sections ofthe sample slide 14 is proportional to the light intensity in thecaptured image, thereby giving rise to a dark field image. Furthermore,it will be appreciated that capturing of a positive image avoidsproblems associated with a high background intensity of a transmittedportion of the illuminating beam. FIG. 7 shows an example image 60,which will be described in more detail below.

Referring now to FIGS. 5 and 5A, a second preferred embodiment of animaging device 46 is shown in which components which are similar oridentical to those in the first embodiment have been numberedaccordingly, and suffixed by an “a”. A carriage or slide tray 12 acarries a sample slide 14 a for analysis of the bound array 102 mountedon the slide. An LED light source including an LED bracket 37 a carryinga linear cluster of LED's emits a narrow beam of light 38 a which isdirected towards the underside of the slide. In the particularembodiment, a linear cluster of five LED's is used having a blueemission wavelength of 490 nm to illuminate a band on the slide having awidth or about 10 mm. A mirror 45 a directs the portion 40 a of theinitial light beam containing rays which have been diffracted orotherwise deflected at bound binding parser in the array towards a linescan capable digital camera device 44 a. In the embodiment, a BaslerL101K line scan camera is used.

A slide tray 12 a which includes a pair of drive tracks 48 is moved toand fro by means of a pair of drive rollers 18 a and idler rollers 70which are rotated using a DC motor 16 a. A push button 78 is pressed topresent the slide tray 12 a for insertion of the sample slide. Thesample slide is inserted into a recess 82 within the slide tray 12 a,with the leading edge of the slide pushing up against a sprung loadedslide retainer 83, and the trailing edge of the slide abutting against aslide stop 85. A slide sensor indicates the presence of the slide bysensing a obturation in the cutout 86 as it is displaced by the slide. Awindow 87 allows the light beam 38 a to illuminate the underside of theslide, and finger aperture 88 enables the slide to be readily removedand replaced.

A slide sensor 89 senses when a slide has been inserted and signals acontroller 50 ato commence acquisition. The motor 16 a advances theslide tray 14 a at a speed sufficient for the line scan camera 44 a totake successive linear images of the light diffracted or otherwisedeflected by the bound cells on the slide it is moved across the band oflight 38 a emitted from the light source 37 a. The line scan camera 44 ahas a linear array of sensors having a width of one pixel and a lengthof 1,024 pixels. In the present embodiment, this equates to a linearimage having a length of 25 mm and a width of 0.025 mm. As a result, asthe slide tray is moved forwards, a successive series of linear imagesof the bound cell array are scanned by the line scan camera 44 a. It isclear from FIG. 5 how non-diffracted or non-deflected light 42 plays norole in the formation of the image as it is reflected away from thelinear aperture of the line scan camera. As the bar code 124 on theslide reaches the imaging region, this is sensed by a microswitch (notshown) as a result of which the light source 37 a is turned off and thelight source 90 is turned on to illuminate the slide bar code, therebyallowing the bar code 124 to be included in the image. A proximitysensor 81 detects the presence of the slide tray and signals theprogrammable logic controller (PLC) 50 a to reverse the drive motor toenable removal of the sample slide 14 a. If detection of fluorescencevia fluorochromes is required, the light source 37 a is turned back on.Full extension of the slide tray is detected by a proximity sensor 80sensing a sensor target 89 on the slide tray, and signaling thecontroller to turn the drive motor 16 a off.

In FIG. 6A, a PLC 50 a which controls the operation of the imagingdevice 46 is shown having as inputs the proximity sensors 80 and 81 andslide sensor 89, together with push button 78. The PLC outputs includesmotor forward and reverse outputs for the DC motor 16 a, and LED outputs48 a and 49 a for indicating scanning and error events respectively.Also included are outputs for controlling the operation of therespective primary and bar code illuminating light sources 37 a and 90,and a frame trigger output 91.

The basic principles of operation of the example imaging devices 10 and46 embodying the present invention are as follows, with reference toFIGS. 1, 2, 5 and 6:

-   -   Developed slides 14 are inserted into the slide carriage 12 and        12A in a horizontal orientation.    -   The slides 14 are transported through the planar light beam 38        to obtain a series of linear images.    -   If the slide 14 has been sequentially scanned successfully, an        LED 48 (FIG. 6) will indicate the scan is complete. If not, an        error indicator 49 and optional audible alarm will be activated.        Diagnostic tips will be displayed on an liquid crystal display        (LCD) display 51 of the machine or a connected PC. Additional        indicators include scanning LED 49.1, “tray full” LED 49.2 for        indicating a full drip tray, and “clean” LED 49.2 indicating        when the imaging device requires cleaning.    -   The slide 14 is ejected from the machine and removed.    -   The linear images are processed in the processing unit 50        (FIG. 2) or an external PC-based frame grabbing card and results        made available to the LCD Display 51 (FIG. 6), a serial port 52,        and/or an Ethernet port 54, or the PC monitor.

In the example embodiment, the bound binding partners are made visibleby transmitting light through the slide and viewing the light diffractedor otherwise deflected at an offset planar orientation by the bindingevents. The image 60 shown in FIG. 7 is assembled sequentially from aseries of linear-images taken using a prototype of the exampleembodiment

Although the image 60 is relatively coarse, a person skilled in the artwill appreciate that there is sufficient information to interpret aresult. It is expected that a much cleaner image can be captured byproduction scanners embodying the present invention.

In the prototype of the example embodiment, the cells are visiblethrough a narrow band across the slide (approximately 2 mm). The slideis moved with respect to the light source and camera, with a thin (0.025mm) slice of image taken for each movement, and then reassembled in theprocessing unit 50, which may form part of the camera. Alternatively,the camera communicates with a frame grabber card in a host computer viaa camera link or data communication cable. Image acquisition andprocessing is performed using appropriate in the computer-base software.

In the following, some further aspects of example embodiments of thepresent invention are discussed.

Image Normalisation and Processing:

Images from scanner to scanner and across time will differ for the samepatient sample due to variation/degradation in the light source, imagingdevice, slide's biology and other environmental conditions. The use ofreference binding partners such as e.g. monoclonal antibodies (Mab's) 62(FIG. 7) in at least the four corners of each array and other outerperipheral regions of the array can be used to normalise the image by:

-   -   Indicating the biological condition of the slide;    -   setting an upper limit for the intensity of each dot and by        measuring the background around the array, an intensity range        can be set from zero to maximum cell binding, and the results        scaled accordingly. This will se any variation/degradation in        the system(s);    -   defining the spatial boundaries for binding partners, and once        found, for pattern recognition of said binding partners

The Mab's 62 man also help locate the dots on each array. Once thecorners of each array are identified, a virtual grid can be overlayed onthe image, locating or other non-reference binding partners. Thebackground can then be removed and the image enhanced using techniquesknown to those skilled in the art. The average intensity of each squarein the grid can then be used to quantify the cell binding on each dot inthe array. A quantification of the resulting scale from e.g. 0-100 orequivalent pixel intensity has been easily achieved. It is expected thatquantification levels well in excess of this should be achievable. Itwill be appreciated by the person skilled in the art, that morereference binding partners, or indeed fewer, could be used in differentembodiments, depending on specific processing requirements. Suchprocessing can be completed either locally or remotely from the cameraand either immediately after image capture or at a later time in thefuture.

FIG. 8 shows a matrix illustrating the digital information that may heobtained from an array for passing on to a pattern-recognition programused to identify a molecular profile.

The following model outlines the example interaction between a scannerembodying the present invention and other devices it communicates with.

The device is preferably able to communicate with:

-   -   External or internal computers and printers, e.g. pathology        computers and printers, and may need to comply with existing        data communication standards and protocols.    -   A reference database.

In the following, examples of other processes preceding and followingthe utilisation of a scanner embodying the present invention will bebriefly outlined for an example implementation.

Processes Preceding the Scanner:

By way of example, the slides are developed by transferring the samples,to the slide and allowing it to incubate, then washing the slide inphosphor buffered saline (PBS) twice, with or without chemical agents toaccess intracellular protein compartments and with or withoutfluorescently active molecules e.g. fluorochrome markers (refer todifferent example sample type described below with reference to FIG. 8),fixing in fomaldehyde and then washing in PBS two or more times.

Processes Following the Scanner:

The slide will be disposed of in biological waste.

The Scanner is Preferably Designed with Suitable Materials and Featuresto Enable:

-   -   External cleaning with a mild detergent.    -   Not to allow contamination of the slide (biological and other).    -   Allow slide receptacle to be cleaned, sterilized and drained of        any fluids that may spill from the slide.    -   Accept a standard glass slide format or equivalent and sustain        in excess of 500,000 operation cycles.

FIG. 9 is a schematic drawing illustrating the optical geometry in animaging device embodying the present invention, for the operation ofdual detection modalities. In this embodiment, a substantially flat beam116 is emitted from an LED light source 110, capable of variablewavelength emissions towards the underside of a sample slide 114. Asdescribed above with reference to FIG. 4, a narrow portion of the lightbeam 116 is transmitted through the slide 114 and emerges as atransmitted portion 118 from the top of the sample slide 114.

In the configuration described in FIG. 9, the sample slide 114 containssamples in which fluorescent markers have been utilised to identify thepresence of a particular molecule, such as e.g. a protein, in thesamples of binding partners. The narrow beam light source 110 is chosenin this embodiment to contain within its spectrum a wavelength suitablefor exciting the fluorescent markers and resulting emission of lightfrom the fluorescent markers. As illustrated in FIG. 9, the lightemitted from the fluorescent markers may be regarded as originating froma point source, thus yielding an omnidirectional light emission field120, creating the series of linear images for subsequent reconstruction.

Accordingly, by way of a mirror element 122, a portion of theomnidirectional fluorescent light emission, indicated as arrow 124, isdirected towards a digital camera device in the form of a line scannercamera capable of monochromic and polychromic detection 126, such as aBasler L301KL. Through appropriate selection of the angular position ofthe mirror 122, it can be ensured that the transmitted portion 118 ofthe illuminating light beam 116 is reflected at the mirror element 122away from the line scanner camera as is shown at 126, i.e. the“collection” angle is offset relative to the transmitted beam portion118.

The optical diffraction/deflection and fluorescent detection, bothrequiring reconstruction of linear images, are directed towards the linescan camera either concurrently or alternatively for each image. Dualdetection modalities are best utilised when the wavelength of lightilluminating from the flat light source approximates to the excitationwavelength of the fluorescently active molecule or molecules. In yetanother embodiment, multiplexing of different fluorescently activemolecules occurs after direction of planar and or offset planar imagesto the camera system, either in an alternative or simultaneousoperational mode, dependent on the polychromatic detection capability ofcamera.

In one embodiment of the invention, the line scan colour camera isarranged to detect diffracted or otherwise deflected light during aforward pass of the slide and fluorescent emissions of light during areverse pass of the slide as it is ejected from the device. The cameraaccordingly operates in a monochromic detection mode on the forward passand converts to polychromic detection mode during the reverse pass. Onthe return pass the slide travels at a slower speed, allowing forgreater exposure time to detect the weaker signal emitted from thefluorochromes. Suitable band pass filter sets, such as Omega filterssupplied by Omega Optical, Inc., and Chroma filters supplied by ChromaTechnology Corp., may be used to ensure that the correct wavelength forstimulating excitation and emission peaks in respect of the selecteddye/nucleic acid complex is used. Preferably though, the wavelength oflight is selected such that detectable excitation is achieved withoutthe need for filters, and using software-regulated discrimination. Inthe present case, the blue LED arrangement is chosen as being suitablefor the celltracker green fluorochrome. It will be appreciated thatbi-colour or tri-colour LED's may be used to provide a broader range ofwavelengths capable of exciting more fluorochromes. Image reconstructionestablishing each binding event is based on one or more digital imagesderived from each detection mode.

The image normalising and processing procedure referred to previouslywith reference to FIGS. 7 and 8 will now be described in more detailwith reference to the flow chart of FIG. 10 and the accompanying images.The imaging device 10 or 46 generates digital data in the mannerpreviously described, as shown as 130. Image grabber software in a framegrabber card forming part of the camera or an external microprocessorconstructs a composite raw image (FIG. 8) from the sequential linearsections, as is shown at 132. Image processing software imports the rawcomposite image, as shown at 134, to generate image intensity values foreach binding event.

The image processing methodology include smoothing the composite imageto remove high frequency variations in pixel intensitics 136 and thenconverting the image to eight levels of greyness on a logarithmic scaleof brightness before generating and displaying a series of concentriclines in the form of a contour map of all intensities, as shown at step138. Smoothing removes high frequency information from the image thatcan make contour lines jagged and leave gaps. The image is converted toeight levels of grey on a logarithmic scale of brightness is formedusing a 256 bit look up table set to convert all image brightness levelsto the following values, namely 1, 2, 4, 8, 16, 32, 64 and 128. Theimage pixel brightness values used as an index to look up this array.The boundaries between the different grey levels are used to use tocreate the contour lines.

The contour map is composed of a background of maximum-intensity pixels,i.e. white pixels having a value of 255. The process traverses the imageby visiting every pixel element inside x and y loops. That clement istreated as the centre of its eight surrounding pixels. If the programfinds a centre pixel with any pixel of the eight that is brighter thanit, it marks that centre pixel as being on a contour. The marking isdone by copying the pixel value (holding only one of the values1,2,4,8,16,32,64,128) to the second array to hold only the pixels of thecontour lines of the image. The result is an array of the samedimensions as the image, with all pixels set to 255, except for pixelson contour lines holding the grey values of the outer edges of regionswhich are darker than their neighbours. A typical contour map derivedfrom the image of FIG. 7 is shown at 140 in FIG. 10A.

The location and identification of all image objects within the contourlines occurs and the confirmation of and enhancements to the circularityare provided, as shown at step 139. In a subsequent normalisation step142, image enhancement associated with the contour lines occurs toremove excessive darkness, thereby further improving circularity andnormalising the image in order to address spatial variations andintensities across the array. In particular, each separate contour lineresulting from the above process is now classified as a separate imageobject. As each contour line is classified, its pixels are added to an“Already processed” array with the same dimensions as the image. Thisallows rapid identification as to whether a given contour line hasalready been processed or not, by looking up any pixel of the line. Thecoordinates of the pixel are used to access the corresponding element inthe “Already processed” array, which was previously cleared.

Every pixel of the contour map array is scanned by an x and y for loops.If a pixel is found to be a member of a new (unprocessed) contour, itscoordinates are passed to the ImageObjectPixelsFind method, which findsthe rest of the pixels on the contour line and classifies the contour asopen or closed.

The method finds all the pixels of the contour line and flags each oneas “Already Processed” in the “Already Processed” array. It also returnstwo other arrays, RegionX( ) and RegionY( ) that hold the correspondingX and Y coordinates of each pixel in the contour line. These two arraysessentially list all the pixels in the contour line. The pixelcoordinates contained in these two arrays are then copied to an array ofimage objects (ie contour lines) which holds a list of all the pixels ineach image object, as well as the number of pixels in the object, andthe grey scale brightness of that object (i.e. of the contour line whichcontains a grey scale region of that logarithmic grey-scale brightness).

The result is that each image object (represented by a contour line) isidentified and listed in the ImageObject( ) array, and can be rapidlytraversed by processing each pixel in the list of pixels for thatobject. If the number of circular objects with contours of more than 45pixels is less than 10 then the image is rejected with the message:

“This image is too poor to process. (Not enough recognisable dots).Perhaps the slide has dried out? Please re-wet and scan again orre-process.”

Image objects are then classified as being circular or not. Theclassification is performed by a method, that analyses a single imageobject. This routine is called in a loop which processes every object onthe image. A routine is used to calculate the length of the contour lineof the object and whether is a closed or open object. Closed means thecontour is a loop. Open means it is a line whose end points do notcoincide. Each object has a contiguous contour line. The routine alsouses an ImageObjectCenter to find the position of the centre of anyclosed object, and also its diameter in the X and Y directions. Usingthe circumference of the object and its diameter information, theroutine then classifies the object as circular or not.

The result is that certain image objects are now classified as circular.These are candidates for the contours of dots on the array. The centresof many of these objects will coincide with the centres of dots on thearray. The process of locating these objects and their centres breaksthe back of the task of locating the array somewhere on the image, andallows the virtual grid to be accurately aligned over it.

The image enhancement routine enhance the image to show only the rangeof brightness that contains significant information. This will enhancethe contrast of the informational parts of the image by removing verydark areas that carry very little information. This will make it easierto detect circular regions indicating dots, and also tends to normalizethe image somewhat to enhance diagnostic accuracy.

One way to discover which brightness levels contain information is tolook for the first logarithmic brightness level that contains anycircular regions (starting at the darkest level). Contour brightnessruns from 1 to 128 in eight steps, doubling in brightness every time.The first level containing at least one suitable circular object islabelled as level n. Then the image is reprocessed to show only theinformation between the Brightness B of level n-1 (B_(n-1)) and 255.This is done by applying the following formula to each pixel of theimage to produce the pixels of the new enhanced contrast image:PixelValue(x,y)_(new)=Max(0, (255*(PixelValue(x,y)_(old) −B_(n-L))/(255−B _(n-1)))))

for all pixel coordinates x,y of the image

Where the Max( ) function simply ensures that no new pixels have anegative brightness value.

The new enhanced image is then completely reprocessed in a second phaseto obtain a new set of contour objects. This includes completelyregenerating the 8 level brightness regions and the contour objects andreclassifying them. However, this enhancement process is obviouslyomitted from the second phase of processing. This step is shown at 146,initial placing of a virtual grid 147 over the image, as is shown inFIG. 10B.

To find the columns of the array, a single histogram is drawn across theimage of the number of circular objects with a x-coordinate falling intoeach histogram cell. Histogram cells are one pixel wide. The histogramcontinues right across the width of the image. When finding columns, they-coordinate of the circular object is ignored. (A similar process isused to find rows, but this time, the histogram is drawn down the array,and the use of x and y-coordinates is reversed.) If the dots fall intocolumns, the histogram will show a peak for each column.

By looking at the peaks and finding the best integral column separationthe location of each column on the grid can be identified, unless thereare simply too few dots in the image to make this possible. Thehistogram requires some smoothing, to allow reliable detection of itspeaks.

If the peaks of the histograms are smeared, or they have closely-spaceddual peaks, the slide image is probably skewed. To aid algorithmevaluation, testing and debugging, it may help to draw the histograms asan overlay of the image. As a result, the position of at least some ofthe rows and columns of the array are known, and the column separationand row separation can be established, allowing a length conversionscale to be developed between the array design and the array image.

Each image object is allocated to one of the identified grid rows ifpossible. Any image object flagged as circular and lying on a validcolumn is allocated to one of the rows found providing it falls withinspecified limits.

Due to various factors, it has been found that it is not safe to assumethat the row skew is the same as the column skew, in other words theskew may not be all due to image rotation, some may be due to otherforms of image distortion. Thus vertical and horizontal skew is removedseparately. However, skew is assumed to be linear and any higher orderdistortions are not corrected.

For each grid column found, the slope of that column is identified byfitting a straight line to the x,y coordinates of the image objectsflagged as falling in that column. This fitting is performed using thestandard least-squares method. The slope of all the columns is averagedto find the average vertical skew for the image. The de-skewing step isillustrated at 150 in the flow chart. Row skew is found and corrected byan analogous method.

Once the image has been de-skewed, image analysis is repeated once moreon the enhanced de-skewed image, recreating image areas and contourlines, and re-classifying the image objects. Obviously the enhancementand de-skewing processes are skipped for this third phase.

Those circular image objects falling on column row intersections withinreasonable boundaries are classified as dots. Other image objects arenot classified as dots and are ignored for the following process.

The previously generated row and column separations are used to generateapproximate X and Y scales (which may well be quite different). Thesescales can then be used to convert image pixels to mm and vice versa sodimensions in the slide definition can be related to dimensions on arraypart of the image, is indicated at step 152 in the flow diagram.

The first and last columns of the slide are used to locate the arraydefinition over that part of the image holding the array. The Xco-ordinates are known from the first and last columns of the array,from which can be worked out the co-ordinates of the remaining interiorcolumns.

An overlay of the image with the virtual grid is displayed, as isindicated at FIG. 10B. The centres of the co-ordinates are located byfinding the nearest image data to each plan corner dot. The enhancedde-skewed image is overlayed to the operator with the deduced corner dotcircled. The operator then has the option to locate the corner dot, inthe event of the corner dot location failing for poor quality images.Once the operator has clicked the final dot, the locations are used toallow the slide plan over the image of the array.

Using the new corner dot positions it is possible to calculate a moreaccurate conversion scale between images pixels and mm. This allows thevirtual grid representing the plan of the array area of the slide to beaccurately located over the image of the array, so each dot appearscentred in its own grid square.

The new x and y-scales to convert from pixels to mm are calculated bycomparing the distances the x-y co-ordinates of the centres of the fourcorner dots in pixels in the appropriate direction, against the mmdistance of these dots on the array plan derived from the SlideDefinition.

The row case is harder because the top and bottom rows of the array areless distinctive than the left and right columns, holding progressivelydiluted antibody subtypes that do not all show clearly on the image.Additionally, a complete top or bottom row may be almost obscured byfluid, spurious light from fluid edges and waves, and/or progressivedrying out of the slide starting at the top or bottom, and not all rowsmay have been identified in the earlier processes, since some rows mayhave almost no dots visible.

Thus a different algorithm is used to align the slide plan over theimage in a vertical direction. Firstly row separations are converted tomm using the approximate scale derived above. Then the slide design isconceptually slid vertically up and down the image, looking for the bestmatch between (a) circular image objects recognised as left and rightedge dots and (b) the edge dots on the plan. This is done in aniteratively in a single pass by conceptually moving the plan down theslide image one pixel at a time, and summing the total distance betweeneach image dot and it's nearest neighbour on the plan. The shortest suchdistance, and its pixel index is recorded during the iteration. When theiteration is complete, this particular location is the best guess foraligning the slide design over the image of the array.

If a dot read error is detected, data entered remains on the screen, andthe operator has to the option to rescan the image, as is shown at 180.

At step 158, background correction calculated using averaged correctedintensity for each dot as identified is performed. Intensity values areread off one by one. As is shown in FIG. 10C, a histogram 162 is formedof the image intensity from 0 to 255 of every pixel in the grid squarelocator containing the dot. The local background intensity is found anddefined as the pixel brightness level M. M divides the pixels in theregion into two sets, namely a dimmer set comprising all the pixelswhich are dimmer or equal in brightness to M, and a brighter set Bcomprising all the pixels in the set that are brighter than M. Thenumber of pixels in set A bears a pre-set ratio to the number of pixelsin set B and is typically 50%.

Having determined which pixels are part of the local background (above),it is now easy to calculate the relative brightness of each dot.

The process starts by summing the image intensities of all the pixelsinside the dot region, based on its calculated radius. The algorithmalso includes consideration of pixels that lie partly inside the dotradius, accumulating a fractional part pro-rata according to thefraction of the pixel lying inside the dot. The number summed (Delta) isthe pixel brightness less the local background brightness calculatedabove.

The average value of this sum per pixel (InnerAverageValue) is thencalculated by dividing the sum by the total number of pixels inside thedot, including fractional parts.

The Dot value is then formed by taking the InnerAverageValue andnormalizing it to represent it as a ratio against the maximum possiblebrightness above the local background.

As indicated at step 164, once the dot values have been identified andincorporated in a matrix such as that illustrated in FIG. 8, thesoftware performs an iterative approach to best match unknown dotpatterns or molecular profiles to a known consensus pattern from alibrary of disease signatures. Graphical and tabulation presentation ofa molecular profile is provided at 166, and the best matched molecularprofile is confirmed against the library of consensus patterns as thebasis for diagnostic or prognostic determinations based on a rankingmethod. The analysis may be repeated at 168 if unsatisfactory, or theimage may be rescanned at 180. Once a matched molecular profile has beenobtained, a diagnostic report is printed and the data is sent to acentralised database, as is shown at 170.

It will be appreciated by the person skilled in the art that numerousmodifications and/or variations may be made to the present invention asshown in the specific embodiments without departing from the spirit orscope of the invention as broadly described. The present embodimentsare, therefore, to be considered in all respects to be illustrative andnot restrictive.

For example, while certain example sample materials have been described,it will be appreciated that the present invention is not limited to theanalysis of particular sample materials. Furthermore, it will beappreciated that the present invention is not limited to use indiagnostic or prognostic applications.

In the claims that follow and in the summary of the invention, exceptwhere the context requires otherwise due to express language ornecessary implication the word “comprising” is used in the sense of“including”, i.e. the features specified may be associated with furtherfeatures in various embodiments of the invention.

1. An imaging device comprising: a carrier stage for carrying a sample slide, a light source for illuminating the sample slide, said sample slide including an array of samples, a drive mechanism for moving the carrier stage and the sample slide relative to the light source such that successive portions of the sample slide are illuminated by the light source; a digital optical camera system arranged to operate in at least two modes, namely a diffraction or deflection mode, in which light rays diffracted or otherwise deflected at the array of samples on the sample slide are captured by a camera, and a fluorescent mode, in which light rays emitted from fluorescent markers on the array of samples are captured: wherein the digital optical camera system is arranged to operate in the deflection or diffraction mode when the drive mechanism moves the carrier stage in a first direction and is arranged to operate in the fluorescent mode when the drive mechanism moves the carrier stage in a second direction; wherein the digital optical camera system is disposed such that, in use, the digital optical camera system captures substantially only said successive portions of light rays which emerge from the sample slide at an offset angle relative to light rays from the light source transmitted through and emerging from the sample slide to generate a series of partial images arranged to be reconstructed into an image of the sample slide or array of samples.
 2. An imaging device according to claim 1 wherein the light source is a linear light source arranged to emit a substantially narrow beam, whereby successive portions of the sample slide that are illuminated are band-like portions, and whereby the series of partial images are linear images.
 3. An imaging device according to claim 1 wherein the digital optical camera system is disposed such that, in use, it receives substantially only light rays which are diffracted or otherwise deflected at said array of samples on the sample slide.
 4. An imaging device according to claim 1 wherein the digital optical camera system includes a discriminator for preventing light rays which are not diffracted or otherwise deflected by the sample array from being captured by the camera system.
 5. An imaging device according to claim 4 wherein the discriminator includes at least one reflector positioned to direct diffracted or otherwise deflected light rays emerging from the sample slide at the offset angle towards an imaging lens of the camera system.
 6. An imaging device according to claim 4 wherein the digital optical camera system includes a line scan camera capable of sensing a linear image.
 7. An imaging device as claimed in claim 1, wherein the digital optical camera system is disposed such that, in use, light rays emitted from fluorescent markers on the sample slide are captured.
 8. An imaging device according to claim 1 wherein the optical camera system is arranged to detect light rays in both the visible and non-visible portions of the spectrum.
 9. An imaging device as claimed in claim 1 wherein the samples comprise cells bound to binding partners on the sample slide.
 10. An imaging device as claimed in claim 1, wherein the imaging device comprises a sampling compartment in which, in use, the carrier stage is located, and an electrical components compartment, wherein the electrical components compartment is fluid sealed from the sampling compartment, whereby, in use, fluid contamination of components inside the electrical components compartment from the sampling compartment is inhibited, the carrier stage including a tray element disposed, in use, underneath the sample slide for collecting fluid spilled from the sample slide.
 11. An imaging device as claimed in claim 1, wherein the imaging device includes an interface unit for interfacing to at least one device selected from a group including an external reference database, an external storage database, an external PC, and an external printer.
 12. An imaging device according to claim 6 wherein the line scan capable camera is a line scan camera adapted to scan linear images having a width of one pixel.
 13. An imaging device according to claim 1 wherein the partial images and the reconstructed images are dark field images. 